These materials are generated by Gerko Vink, who holds the copyright. He thanks many people, as the thoughts and code in these slides are the result of years-long development cycles. When someone has contributed to the content of the slides, he has credited their authorship.
The intellectual property belongs to Utrecht University. Images are either directly linked, or generated with StableDiffusion or DALL-E. There is no information here that exceeds legal use of copyright materials in academic settings, or that should not be part of the public domain.
Warning
You may use any and all content in this presentation - excluding names - and submit it as input to generative AI tools, with the following exception:
You must ensure that the content is not used for further training of the model
We have limited time and so much to talk about. Today we therefore focus on creating reproducible research compendiums with Quarto and developing R packages with RStudio.
We will not focus on programming. After all, programming you learn by doing, not by listening. If you’d like to learn more about programming, I suggest you take a look at the following resources: https://www.gerkovink.com/rijkR/. This is a different version of this course, but it contains a lot of useful information.
That said, let’s at least cover the programming basics.
R Programming basics
classes in R
vectors, matrices, arrays, data frames, tibbles
pipes
|>, %>%, %$%, %T>%, %<>%.
future
purrr and the map/Reduce workflows
ggplot2 and plotly
targets (formerly drake)
rng’s and set.seed()
This lecture
Reproducibility
Research compendiums
Quarto
Why bother?
We would like our results to be as fully reproducible as possible:
A. Reproducibility is one of the pillars of science
It is the standard by which to judge scientific claims
It helps the cumulative growth of knowledge - no duplication of effort
B. Reproducibility may greatly benefit you
You’ll develop better work habits
Better teamwork - especially with new team members
Changing or amending your work is much easier
Higher research impact - more likely to be picked up and cited
By the end of today…
… you will have heard what reproducible research and programming is and why it is important
… you will be able to produce a reproducible research archive from scratch
… you’ll have published an open R package with proper unit tests
… and much more!
Definition
A result is reproducible when the same analysis steps performed on the same dataset consistently produces the same answer.
Reproducibility of R scripts
Reproducible research is not the norm:
74% of R files failed to complete without error
Research compendium
Definition
“A research compendium is a collection of all digital parts of a research project including
data,
code,
texts
…
The collection is created in such a way that reproducing all results is straightforward”1
“The compendium serves as a means for distributing, managing, and updating the collection”2
title, author, and date provide metadata about the document.
output specifies settings related to the document’s output format and theme.
The YAML header allows you to control how the document is rendered and presented. It ensures that important document information and settings are stored in a human-readable and structured format at the beginning of the document.
Text in Quarto
Text is text. Nothing more, nothing less
Header
Subheader
Subsubheader
# This is a heading indicating a section## This is a heading indicating a subsection### This is a heading indicating a subsubsection
But in the above I used
### Header#### Subheader##### Subsubheader
Figures
A surfing elephant
The above figure is left-aligned The caption is also left-aligned
A surfing elephant
The above figure is centered The caption is still left-aligned
Let’s assume that \(y\) is a vector with \(N\) elements such that \[y \sim \mathcal{N}(\mu, \sigma^2),\] where we set in our simulations \(\mu = 10\) and \(\sigma^2 = 5\). We do something for every \(y_i\) with \(i = 1, \dots, N\).
Code chunks
We already saw some markdown code chunks. We can run inline code to calculate that 12 * 234 is 2808 by using `r ` and code chunks to evaluate larger blocks of code, such as:
Code
library(ggplot2)ggplot(mpg, aes(x = hwy, y = cty, color = cyl)) +geom_point(alpha =0.5, size =2) +scale_color_viridis_c() +theme_minimal()ggplot(mpg, aes(x = hwy, y = cty, color = displ)) +geom_point(alpha =0.5, size =2) +scale_color_viridis_c(option ="E") +theme_minimal()
(a) Color by number of cylinders
(b) Color by engine displacement, in liters
Figure 1: City and highway mileage for 38 popular models of cars.
The previous slide
## Code chunks and explicit caching```{r}#| output: asisanswer <-12*234```
We already saw some `markdown` code chunks. We can run inline code to calculate
that `12 * 234` is ` r answer ` by using `r ` and code chunks to evaluate
larger blocks of code, such as:
```{{r}}#| output: asis#| label: fig-mpg#| fig-cap: "City and highway mileage for 38 popular models of cars."#| fig-subcap:#| - "Color by number of cylinders"#| - "Color by engine displacement, in liters"#| layout-ncol: 2#| column: page#| code-fold: truelibrary(ggplot2)ggplot(mpg, aes(x = hwy, y = cty, color = cyl)) + geom_point(alpha = 0.5, size = 2) + scale_color_viridis_c() + theme_minimal()ggplot(mpg, aes(x = hwy, y = cty, color = displ)) + geom_point(alpha = 0.5, size = 2) + scale_color_viridis_c(option = "E") + theme_minimal()```
Explicit Caching
technique used in RMarkdown / Quarto to reuse the results of expensive steps
improves efficiency of document generation and preserves reproducibility
Why Use it?
Efficiency: Caching saves time and resources by avoiding repeated execution.
Reproducibility: ensures the same results are used when rendering the document, even if the code is modified or re-executed.
Version Control: Caching reduces the size of version-controlled documents by storing intermediate results separately.
How Does Explicit Caching Work?
In RMarkdown and Quarto, you can use cache directives to specify which code chunks should be cached and which should not.
Cached chunks are identified by a unique hash based on their code content, ensuring consistency.
# A tibble: 748 × 9
age hgt wgt bmi hc gen phb tv reg
<dbl> <dbl> <dbl> <dbl> <dbl> <ord> <ord> <int> <fct>
1 0.035 50.1 3.65 14.5 33.7 <NA> <NA> NA south
2 0.038 53.5 3.37 11.8 35 <NA> <NA> NA south
3 0.057 50 3.14 12.6 35.2 <NA> <NA> NA south
4 0.06 54.5 4.27 14.4 36.7 <NA> <NA> NA south
5 0.062 57.5 5.03 15.2 37.3 <NA> <NA> NA south
6 0.068 55.5 4.66 15.1 37 <NA> <NA> NA south
7 0.068 52.5 3.81 13.8 34.9 <NA> <NA> NA south
8 0.071 53 3.89 13.8 35.8 <NA> <NA> NA west
9 0.071 55.1 3.88 12.8 36.8 <NA> <NA> NA west
10 0.073 54.5 4.2 14.1 38 <NA> <NA> NA east
# ℹ 738 more rows
1. Dependency Isolation: renv creates a dedicated environment for each project, preventing conflicts between package versions.
2. Reproducibility: With renv, you can capture and record the specific package versions used in your project, ensuring reproducibility over time.
3. Collaboration: Share your project with others, and they can easily recreate the same environment using renv.lock.
4. Easier Package Installation: renv simplifies package installation. Just run renv::restore() to set up the project environment.
5. Automatic Snapshot: renv automatically generates a renv.lock file, listing all package versions used, making it easy to recreate the environment.
6. Compatibility: renv is compatible with popular version control systems like Git, facilitating collaboration and sharing.
Use renv::snapshot() to update the renv.lock file when you add or update packages. You can use renv::activate() and renv::deactivate() to activate or deactivate renv for your project.
Diagrams
```{{mermaid}}%%| echo: falseflowchart LR A[Hard edge] --> B(Round edge) B --> C{Decision} C <--> D[Result one] C --> E[Result two]```
flowchart LR
A[Hard edge] --> B(Round edge)
B --> C{Decision}
C <--> D[Result one]
C --> E[Result two]
To use rticles from RStudio, you can access the templates through File -> New File -> R Markdown. This will open the dialog box where you can select from one of the available templates:
Quarto
The quarto use template command can be used to create an article from one the below formats.
Terminal
quarto use template quarto-journals/acmquarto use template quarto-journals/plosquarto use template quarto-journals/elsevierquarto use template quarto-journals/acsquarto use template quarto-journals/jss
Citations
Oberman and Vink (2023) and Cai, Van Buuren and Vink (2023) are some recent publications. Some references - such as some work by Reinder Banning and Gerko Vink are much older (2010) or contain simple and contemporary solutions by Volker and Vink (2022, 2–4) or cool but potentially confusing images (Schouten and Vink 2021, 1255).
Banning, Reinder, and Gerko Vink. 2010. “Methoden En Indicatoren Voor Het Evalueren En Beschrijven van Controle- En Correctieprocessen.” CBS nota DMV-2010- 06-08-RBNG-GVNK. Centraal Bureau voor de Statistiek.
Cai, Mingyang, Stef van Buuren, and Gerko Vink. 2023. “Graphical and Numerical Diagnostic Tools to Assess Multiple Imputation Models by Posterior Predictive Checking.”Heliyon.
Oberman, Hanne I., and Gerko Vink. 2023. “Toward a Standardized Evaluation of Imputation Methodology.”Biometrical Journal, 2200107. https://doi.org/10.1002/bimj.202200107.
Schouten, Rianne Margaretha, and Gerko Vink. 2021. “The Dance of the Mechanisms: How Observed Information Influences the Validity of Missingness Assumptions.”Sociological Methods & Research 50 (3): 1243–58.
Volker, Thom, and Gerko Vink. 2022. “Synthetic Data in r: Generating Synthetic Data with High Utility Using Mice.”Https://Www.gerkovink.com/miceVignettes/Synthetic/Synthetic.html.
## Citations
@eval and Cai, Van Buuren and Vink [-@cai2023graphical] are recent publications. Some references - such as some work by Reinder Banning and Gerko Vink are much older [-@banningvink] or contain simple and contemporary solutions by @vigntg [p. 2-4] or cool but potentially confusing images [@schouten2021dance, 1255].
::: footer
But you can also refer to @eval here
:::
::: {#refs}
:::
